The present invention relates generally to the field of eyetracking and, more particularly, to a system and method for analyzing eyetracker data collected from a user viewing the display of dynamic hypermedia pages.
In various fields, psychologists and other researchers have used eyetracking devices (or eyetrackers) to record eye fixations, eye movements, and pupil dilation, as a way to infer the locus of a person""s visual attention. This inference is generally referred to as the xe2x80x9cmind-eye assumptionxe2x80x9d. In this regard, disclosures related to eye fixations, movements and dilation are in U.S. Pat. No. 5,471,542 to Ragland, U.S. Pat. No. 5,410,376 to Cornsweet et al., and U.S. Pat. No. 5,231,674 to Cleveland et al. In one application, these types of systems have been utilized to monitor eye-movements to gather data from television viewers. U.S. Pat. No. 4,789,235 to Borah et al. discloses a method and system for generating a description of the distribution of looking time as people watch television commercials. This reference describes a system in which eyetracking is used to measure individual""s interest in distinct visual areas of different television scenes.
A basic problem in eyetrackers concerns the mapping of the eyetracking data (xe2x80x9cpoints-of-regardxe2x80x9d of the eye) onto elements of the hypermedia pages to thereby identify the elements upon which the user was fixated/focused upon (xe2x80x9celements-of-regardxe2x80x9d). The mapping of points-of-regard onto elements-of-regard is referred to herein as the xe2x80x9cpoints-to-elementsxe2x80x9d mapping and this mapping poses a significant difficulty in applying eyetracking technology to hypermedia. Generally, eyetrackers monitor and record the user""s eye gaze in a coordinate system that is an overlay of a section of physical space. For instance, an eyetracker might record a set of (x,y) coordinates that identifies a section of space that corresponds to the display screen of a computer. If the information elements displayed on the screen is fixed, then analysis programs can easily determine the relationship of the recorded eye gaze data to the displayed elements.
However, the very nature of hypermedia results in very frequent changes in the user""s display screen, thereby greatly limiting the application of present eyetracking technology. Typically, the user must be restricted in the actions he/she performs. For instance, the use of scroll bars would be disallowed since their use would instantly change the position of the elements and thereby render the recorded coordinates inaccurate. Otherwise, researchers must tediously analyze the eyetracking data manually from video recordings of the user to infer the relationship of eyetracker coordinates to the content that had been displayed or templates describing the location of display elements must be changed by hand every few seconds. Consequently, eyetracking research on hypermedia such as the WWW requires a time investment that is too high for hypermedia experts.
This hand analysis can be facilitated by providing a scene by scene (or screen by screen) data file or by providing tools that permit the analyst to define and label spatial areas corresponding to areas of displayed content. In the Borah patent noted above, eyetracking is used to measure individual""s interest in distinct visual areas of different television scenes. Point of gaze data is compared with a scene by scene data file to produce a third data file showing a mean distribution of viewer looking time for each selected scene. However, in Borah""s invention, the scene data file must be produced manually and thus, a very significant amount of manual effort is required and there is no access to the semantic content of the regions. Analogously, a template of defined content areas can be specified for each display state, for example for each and every view of WWW pages visited by a user to thereby allow more expedited data processing. However, as the number of screen changes increases, so does the number of templates required for analysis. In the case of a Web browser, a new template would be required for each page, and, in fact, for each possible position of the scroll bar for each page. Thus, such solutions is not practical or economical.
Recently, a method has been proposed for presenting high level interpretations of eye tracking data which is correlated to saved display images in U.S. Pat. No. 6,106,119 to Edwards. The Edwards patent discloses a method that stores eye tracking data and correlated display scenarios based on a predetermined condition such as time interval, tracking sequence, a scrolling detection or other activity. Edwards teaches the conversion of the eye tracking data into high level interpretations and assigning a valuation vocabulary to these interpretations. The stored display scenarios are then displayed together with the valuation vocabulary. Thus, the invention of Edwards discloses superimposing eyetracking data over screen shots and teaches techniques for determining when to capture the screen shots. Moreover, the invention of Edwards addresses applicability to hypermedia such as webpages and addresses the problems caused by scrolling by detecting scroll events and taking screen shots before and after the scrolling by the user.
However, the invention disclosed in Edwards reference has been found to only provide a limited advantage to the known prior art eyetracking methods. As previously noted, because the output provided in Edwards is interpreted eyetracking data superimposed over screen shots, a human analyst must then perform the points-to-elements mapping by observing and analyzing the screen shots with the eyetracking data superimposed thereon to identify what elements are beneath the eyetracking data. Thus, a significant amount of human effort is still required to obtain the element-of-regard information. Moreover, because the screen shots as taught in the Edwards reference are merely graphical representations, no additional data processing can be made with the data obtained. For instance, in accordance with the teachings of Edwards, a human analyst would be required to determine a user""s eye movements to a particular element, image, link, text or formatted text, etc. Edwards does disclose use of templates that can aid in identifying the elements during the analysis of the eyetracking data. However, a significant amount of human effort must be expended to create such templates and a new template would be needed for every screen displayed which makes the proposed solution unworkable in a hypermedia context where the screen displayed changes constantly.
All documents cited herein, including the foregoing, are incorporated herein by reference in their entireties.
The present invention provides an improved system and method for analyzing eyetracker data collected from a user viewing display of dynamic hypermedia pages. Additionally, with the present invention human analyst interaction and effort is minimized and points-to-element mapping can be substantially automated.
The system and an information storage media for analyzing eyetracker data collected from a user viewing a display of dynamic hypermedia pages through a browser having an object model interface includes an eyetracker, a logging tool and a mapping tool. The eyetracker monitors location and time of the user""s gaze, and stores eyetracker data of the location and time of the user""s gaze in an eyetracker data file. The logging tool monitors the displayed pages and stores content of the displayed pages into a memory. The mapping tool receives the eyetracker data from the eyetracker data file, processes the eyetracker data into fixation data indicative of the user""s interest, and restores the displayed pages from the memory. The mapping tool also maps the fixations data onto the restored pages to thereby identify elements-of-regard upon which the user fixated and locations thereof, and stores the identified elements-of-regard and the locations thereof in an elements-of-regard database. The fixation data is at least one of information regarding fixation points at which the user""s gaze becomes fixated, information regarding an area proximate to a fixation point that is contained within a bounding box, or information regarding a transition which is a cluster of fixation points related to one another by a predetermined algorithm. The mapping tool may also accesses the object model interface of the browser to identify elements of the restored pages and stores the identified elements of the restored pages in an elements database. The logging tool may also monitor and store events and time data of the events that cause a change in the pages displayed into an event log, and may restore the displayed pages from the memory and the events data in the event log by reenacting the events of the event data on the restored pages. The mapping tool may also time synchronize the fixation data to the events data in the event log, and may calibrate a coordinate system of the eyetracker data stored in the eyetracker data file to a screen coordinate system used in viewing the display of dynamic hypermedia pages. The logging tool and the mapping tool may be software programs that are stored in an information storage media.
In accordance with another embodiment, method for analyzing data collected from a user viewing a display of dynamic hypermedia pages through a browser having an object model interface is provided including monitoring location and time of the user""s gaze, storing the location and time as eyetracker data, processing the eyetracker data into fixation data indicative of the user""s interest, monitoring content of the displayed pages and storing content of the displayed pages into a memory, restoring the displayed pages from the memory, mapping the fixations data onto the restored pages to thereby identify elements-of-regard upon which the user fixated upon and locations thereof, and storing the identified elements-of-regard and the locations thereof in an elements-of-regard database. In accordance with an embodiment, the method also includes accessing the object model interface of the browser to identify elements of the restored pages and storing the identified elements of the restored pages in an elements database. The method may also include monitoring and storing events and time data of the events that cause a change in the pages displayed as event data. The displayed pages may be restored from the memory and the events data by reenacting the events of the event data on the restored pages. The present method may also include time synchronizing the fixation data to the events data as well as calibrating a coordinate system of the eyetracker data to a screen coordinate system used in viewing the display of dynamic hypermedia pages.
In accordance with still another embodiment of the present invention, a system and information storage media for analyzing eyetracker data collected from a user viewing a display of dynamic hypermedia pages through a browser having an object model interface is provided including an eyetracker, a logging tool, and a mapping tool. The eyetracker monitors location and time of the user""s gaze, and stores eyetracker data of the location and time of the user""s gaze in an eyetracker data file. The logging tool monitors the displayed pages and stores content of the displayed pages into a memory. The mapping tool receives the eyetracker data from the eyetracker data file, processes the eyetracker data into fixation data indicative of the user""s interest, and restores the displayed pages from the memory. The mapping tool also maps the fixations data onto the restored pages. The mapping tool accesses the object model interface of the browser to identify elements of the restored pages and stores the identified elements of the restored pages in an elements database. The fixation data is at least one of information regarding fixation points at which the user""s gaze becomes fixated, information regarding an area proximate to a fixation point that is contained within a bounding box, or information regarding a transition which is a cluster of fixation points related to one another by a predetermined algorithm. The logging tool may also monitor and store events and time data of the events that cause a change in the pages displayed into an event log. The mapping tool may restore the displayed pages from the memory and the events data in the event log by reenacting the events of the event data on the restored pages. In other embodiments the mapping tool may also time synchronize the eyetracker data to the events data in the event log and also may calibrate a coordinate system of the eyetracker data stored in the eyetracker data file to a screen coordinate system used in viewing the display of dynamic hypermedia pages.
In accordance with yet another embodiment of the present invention, a method for analyzing data collected from a user viewing a display of dynamic hypermedia pages through a browser having an object model interface is provided including monitoring location and time of the user""s gaze, storing the location and time as eyetracker data, processing the eyetracker data into fixation data indicative of the user""s interest, monitoring content of the displayed pages and storing content of the displayed pages into a memory, restoring the displayed pages from the memory, mapping the fixations data onto the restored pages, accessing the object model interface of the browser to identify elements of the restored pages, and storing the identified elements of the restored pages in an elements database. The method may also include monitoring and storing events and time data of the events that cause a change in the pages displayed as event data. The displayed pages may be restored from the memory and the events data by reenacting the events of the event data on the restored pages. In addition, the method may also include time synchronizing the fixation data to the events data and calibrating a coordinate system of the eyetracker data to a screen coordinate system used in viewing the display of dynamic hypermedia pages. (Use the term xe2x80x9cmayxe2x80x9d when the statement could refer to a dependent type claim.)